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Executive Summary 



As a follow-up of previous efforts to utilize computers in the language 
arts area, the Delaware Department of Public Instruction with the assistance 
of Research for Better Schools initiated a program to support the use of 
computers to improve writing skills of middle level students. The project 
was implemented in each of sixteen school districts throughout Delaware. 
One teacher from each district at the middle .'.evel used the program with all 
of his/her classes • Training and technical assistance were provided to 
involved schools and teachers to facilitate appropriate hardware and 
software acquisition and use with the writing process. Monitoring 
activities were conducted to aid in implementation of the program at each 
site. Evaluation activities were undertaken to assess first year results. 

The computer writing project had the following four goals: 

• to improve student writing skills through the use of computer- 
assisted instruction for the teaching of the writing process 

• to improve student enjoyment of writing 

• to improve student enjoyment of computer-assisted instruction 
for writing 

• to identify teacher needs with regard to implementing key 
elements of the instructional program. 

The project goals relating to implementation and effectiveness concerns 
were translated into process and outcome evaluation questions to be 
addressed in the evaluation design. A pre-test post-test control group 
design was utilized. Each participating school district was not only to 
provide an experimental group and teacher but also a comparable control or 
comparison group where possible. 

Program implementation was assessed through readiness checklists, 
training evaluation forms ♦ classroom observations, and teacher feedback. 
Student writing skills for both experimentalr and controls were tested 
through use of pre- and post-writing samples, analytically scored. A 
student survey was used to measure enjoyment of the writing process and the 
use of computers in writing instruction. 

The evaluation questions and the findings for each are summarized in 
the figure on the following page. It is clear that the program achieved 
remarkable success in its first year of implementation. Students 
significantly gained in writing skills, enjoyment of writing, and in 
enjoyment of computer-assisted instruction. Teachers also enjoyed using the 
program. Recommendations are made fcr further evaluative study to assess 
the maximum potential of the program, to investigate the barriers to 
successful implementation, to study teacher changes in instructional style, 
and to study student changes in learning style. 
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Evaluation Questions and Findings 



Questi ons 

Orientation - Was adequate 
orientation and assistance 
provided for planning and 
selection of hardware/software? 

Selection and Acquisition - Was 
the appropriate hardware and 
software ordered and delivered 
on time? 

Training - Was adequate inservice 
training in computer literacy, 
the writing process, and the 
use of computers in teaching 
writing provided to Project 
Teachers? 



Findings 

YES - Orientation and assistance 
was highly rated by those schools 
and teachers taking advantage 
of what was offered. 

YES - The majority of schools 
ordered and received the 
appropriate hardware and 
software on time. 

YES - Inservice training was 
highly rated by those schools 
and teachers taking advantage 
of what was offered. 



Implementation - Did the teachers 
start-up on time and implement 
the key program elements (writing 
process approach, computer use, 
mini lessons, conferencing, 
monitoring) ? 

■ Writing Skills - Were student writing 
skills enhanced as a result of 
participation in the Project? 



YES - Most teachers started the 
program on time and implemented 
all I ?sy elements . 



YES - Across schools, experimental 
students scored significantly 
higher than controls on writing 
assessment total score and three 
of four sub-scores: organization, 
development, conventions » No 
significant difference was found 
for the focus sub-score (analyses 
of covariance) . 



Enjoyment of Writing - Did students 
get more enjoyment of writing as 
a result of participation in the 
Project? 



Enjoyment of Computer Learning - 
Did students enjoy using the 
computer for learning and 
practicing writing skills • 
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- Individual schools varied 
considerably. But within schools, 
writing assessment total score and 
each of the four subscores, showed 
experimental groups out-gaining 
controls in at least 10 of the 16 
school sites (gain score 
comparisons) 

YES - Both experimentals and 
controls across schools gained 
significantly (correlated t-test). 
But experimentals gained more than 
twice as much (gain score 
comparison) • 

YES - Both experimentals and 
controls across schools gained 
significantly (correlated t-test). 
But experimentals gained more than 
five times as much (gain score 
comparison) • 
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Introduction 

For the past several years, the Delaware Department of Public 
Instruction (DPI) has been making a concerted effort to Increase the use 
of computers for Instruction In traditional subject areas. During the 
1987-1988 school year DPI, with the assistance of Research for Better 
Schools (RBS), pilot tested an Inservlce program for language arts and 
social studies teachers designed to facilitate their Increased use of 
computers for Instruction. The results of the pilot provided evidence that 
computers are effective tools for assisting Instruction. The results also 
seemed to Illustrate the many barriers and pitfalls to overcome In 
Implementing a successful computer-assisted Instruction program. 

The following year, It was decided to extend this Initiative Into the 
area of writing Instruction. A program was established to support the use 
of computers to Improve the writing skills of middle level students. The 
lessons learned from the computer pilot test of the prior year's inservlce 
program were Incorporated into the planning of a new effort Specifically, 
in February 1989, the DPI proposed a project to support and study the use of 
computers in the classroom to assist teachers in their instruction of the 
writing process at the middle school level. The project v.as to be 
Implemented in each of sixteen school districts throughout Delaware. 
Writing teachers in each of the selected middle schools (one per district) 
were to participate. The DPI requested assistance from Research for Better 
Schools in carrying out several activities essential to the study. RBS was 
to assist the DPI in providing staff development and technical assistance 
and was to conduct an evaluation of the project. 

This report summarizes the evaluation activities for the Delaware 
Middle Level Computer Writing Project. It contains four sections and an 
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appendix. The remainder of this section describes the Project goals and the 
program elements. The second section presents an overview of the evaluation 
design and procedures carried out by RBS in conducting its evaluation study. 
The third provides a brief description of the results of these evaluation 
activities. The final secticr presents a summary of evaluation procedures 
and findings, along with some conclusions and implications. The appendix 
contains copies of key project documents and data collection instruments. 
These include: 

• Memorandum of Agreement 

• Schedule of Activities foi- Delaware Middle Level 
Writing Project 

• Guidelines for Selection of Control or Comparison 
Group 

• Agenda for April and May Orientation Sessions 

• Readinesfj Checklist for Participation in the Middle Level 
Computer-Writing Project 

• Teacher Inservice Evaluation Form 

• School Hardware /Software Decisions Form 

• Telephone Readiness Checklist 

• Software Training Evaluation Form 

• Classroom Observation Form 

• Teacher Log 

• Writing Assessment and Scoring Procedures 

• Pre-Writing Assessment, student Survey #1, Instructions 

• Post-Writing Assessment, Student Survey #2, Instructions 



Pro.ject Goals 

The design of the computer writing project was guided by four goals. 
These goals were as follows: 

• to improve student writing skills through the use of computer- 
assisted instruction for the teaching of the writing process 

• to improve student enjoyment of writing 

• to improve student enjoyment of computer-assisted instruction for 
writing 

• to identify teacher needs with regard to implementing the five 
elements of the instructional program. 

Additionally, it was anticipated that the Project would have a positive 
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impact on teachers in their classroom instruction* These four goals guid«d 
the development of the evaluation design (see the Design and Procedures 
section of this report). 
Key Elements 

The program had five principal components or key elements. These were? 

• teacher daily use of the writing process approach to teaching 
writing, focusing on the stages of pre-writing, drafting, and 
revising 

• teacher daily use of all computers for students, on a rotating 
basis, to work through their writing assignments 

• teacher daily use of a short writing lesson or mini-lesson, thus 
allowing students to spend most of their class time working on 
writing assignments 

• teacher daily use of conferencing, with individuals or small groups 
of students, to encourage students' writing (as author, rather than 
as critic) 

• teacher on-going monitoring of students working on the computer and 
on writing assignments, to be aware of their progress and needs for 
assistance. 

Each of the five components of the program needed to be carried out 
effectively if the desired outcomes were to be realized. 

Design and Procedures 
A total of sixteen middle schools participated in the computer writing 
program, one classroom in each of the sixteen districts in Delaware offering 
middle level education. An experimental teacher was selected to teach the 
computer writing program to his/her classes at each school. In addition, 
each school was to provide a comparable control or comparison group where 
possible. The control group could consist of: (a) a comparable teacher and 
classes within the same school, (b) all other students (non-experimental) at 
that grade level within the same school, or (c) a teacher and students at a 
different but comparable school. Thirteen of the sixteen participating 
schools were able to provide control groups for the evaluation study. These 



control groups were selected on the basis of comparability of teachers and 
students and on the expectation that the control students would have limited 
exposure to the use of a computer in learning writing skills. Experimental 
and control group students were administered the same evaluation instruments 
on a pre- and post-test basis. 

Each of the sixteen districts that participated in the Delaware Middle 
Level Computer Writing Project was required to sign a Memorandum of 
Agreement acknowledging the specific conditionb required for participation. 
During the spring of 1989, staff from the DPI and RBS attended several 
meetings to plan for the Project's implementation and evaluation. One of 
the outcomes of these planning sessions was the development of a timeline 
for Project activities. 

The Project was designed with four stages in mind, each of which was to 
be evaluated. They were: 1) orientation of participating teachers, 
2) selection and acquisition of hardware and software, 3) training of 
participating teachers, and 4) implementation. The first two stages were 
completed during the spring and summer of 1989. Then, district personnel 
selected the hardware and software needed for implementation and teachers 
began, early in the school year, to use the computers in their daily 
instructional activities. Throughout the school year, teachers were 
monitored and were provided with technical assistance and support as needed. 
Evaluation Desig n 

The Project goals were incorporated into a set of four process and 
three outcome evaluation questions which formed the framework for the 
evaluation design. This framework is outlined in Figure 1. 

The process component of the evaluation was to focus on insuring that 
prerequisite conditions for proper program implementation were met. It was 
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Figure 1 



EVALUATION DESIGN 
Process Ei;aluation 

Evaluation Question Measure Expectation 



Orientation - Was adequate 
orientation and assistance 
provided for planning and 
selection of hardware /software? 

Selection and Acquistion - Was 
the appropriate hardware and 
software ordered and delivered 
cn time? 

Training - Was adequate inservice 
training in computer literacy, 
the writing process, and the 
use of computers in teaching 
writing provided to Project 
teachers? 

Implementat ion - Did the teachers 
start-up on time and implement the 
key program elements (writing 
process approach, computer use, mini- 
lessons, conferencing, monitoring}? 



• Readiness Checklist 

• Teacher Inservice 
Evaluation Form 



• School Hardware/Software 
Decisions Form 

• Telephone Readiness 
Checklist 

• Software Training Evalua 
tion Form 

• Teacher Inservice 
Evaluation Form 



• Telephone Readiness Checklist 

• Classroom Observation Form 

• Teacher Log 

• Teacher Feedback 



• Mean overall rating of 4 or 
above on a 5-point Likert 
scale. 

• Training needs identified. 

• Necessary and appropriate 
hardware / software , selected 
ordered, and received by 
September. 

• Mean overall rating of 4 or 
above on a 5-point Likert scale. 

• Any outstanding needs to be 
met by September 1. 



• All training hardware, software 
needs satisfied. 

• Program on schedule for start- 
up in September. 

• Regular use of key program 
elements in classroom by all 
teachers. 



11 

ERIC 



EVALUATION DESIGN (continued) 



Outcome Evaluation 



Evaluation Question 

Writing Skills • Were student 
writing skills enhanced as a 
result of participation in 
the Project? 



Measure 

• Pre- and Post-Writing 
Assessment 



Expectation 

Significantly greater 
writing proficiency in 
program group over comparison 
group. 



Enjoyment of Writing - Did 
students get more enjoyment of 
writing as a result of 
participation in the Project? 



Enjoyment of Computer Learning 
Did students enjoy using the 
computer for learning and 
practicing writing skills? 



• Pre- and Post-Student 
*^urvey 

• Teacher Anecdotes 



• Pre- and Post-Student 
Survey 

• Teacher Anecdotes 



Significantly enhanced enjoy- 
ment of writing by self-report 
in program group over 
comparison group • 
Teacher reports of students* 
enhanced enjoyment of writing. 

Student responses indicate 
enjoyment in using the 
the computer for writing 
assignments • 

Teacher reports of students* 
enjoyment of using the 
computer for writing. 
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therefore designed to study the course of the four stages of the Project » to 
document their progress^ and to assess the quality of their associated 
activities. Feedback from the process evaluation was to be used by Project 
administrators and staff to adjust and fine tune Project activities in order 
to make the activities more effective. 

The outcome component of the evaluation was designed to assess the 
effects of the Project on student writing skills and on student attitudes 
toward writing and toward using the computer for writing. A pre«-test-> 
post- test control group design was used to enhance the validity of the 
findings* Guidelines for selecting the control or comparison group for each 
Project class included specifications that the two groups be as similar as 
possible on key characteristics (i.e., type of students achievement levels» 
demographic variables). Feedback from the outcome evaluation was to 
document overall program effectiveness and to provide information useful in 
making decisions about Project expansion or replication. 
Procedures 

Figure 1 describes the process and outcome evaluation design in terms 
of evaluation questions, measures, and expectations for each of the four 
stages of the Project. The evaluation procedures associated with each 
evaluation question are described in detail below. The results of these 
procedures are presented in the Results section of the report. 

Orientation. The sixteen Project teachers and several of their school 
administrators participated in a two-day workshop in the spring of 1989. 
The workshop was designed to provide an orientation to the Project, and an 
overview of writing software, computer hardware, and approaches for using 
the computer to assist with writing instruction. During the first 
orientation session, teachers were administered the Readiness Checklist to 



determine their district/ school • s readiness to participate in the Project. 
At the conclusion of the second day of the orientation workshop, 
participants were administered the Inservice Evaluation Form to obtain 
feedback on the orientation sessions and also to assess their readiness to 
select the necessary hardware and software for the Project. 

On May 1, 1989, an orientation session was held for principals and 
computer coordinators from tha sixteen middle schools. This session was 
designed to provide administrators with an overview of the Project and the 
required implementation tasks (e.g., to participate in hardware and software 
selection) and to address any concF.rns or questions that might arise prior 
to implementation. This acti^'lty was obse -ved and evaluated by RBS staff. 

Selection and Acqu isition. RBS constructed a School Hardware /Software 
Decisions Form to be completed by administrators in each district in 
conjunction with the Project teacher, by June 1, 1989. The form asked for 
specifications on the type of computer hardware and software package (s) the 
districts decided to purchase for the Project. 

In terms of the acquisition of the materials ordered, RBS teleohoned 
each of the principals of the participating schools and used a Telephone 
Readiness Checklist to identify staff readiness to implement the Project. 
Information was collected on whether the hardware and software ordered had 
been received, set-up, tested, and was ready to be used for instruction. 

Training. The Inservice Evaluation Form, discussed earlier, served as 
a vehicle for identifying ieacher need for further training on compute- 
hardware, software, or on the writing process. To respond to these 
identified needs, two workshops were conducted at Delaware State College 
during the week of August 21-25, 1989. The first workshop, conducted by 
the DPI, was a fifteen hour course on the Delaware Writing Process for which 
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teachers received one inservlce credit. 

The second workshop consisted of fifteen hours of training on the 
specific hardware and software teachers selected to install for use in the 
Project. This latter workshop was conducted by DPI staff, RBS staff, and 
representatives from computer vendors. At the conclusion of the workshops, 
RBS administered a Software Training Evaluation Form to assess the 
effectiveness of the training on the various hardware and software packages 
and to identify additional needs, if any. 

Implementation . The implementation of the Project was addressed by the 
process and outcome evaluation components. The process evaluation focused 
on monitoring teacher use of the five program elements presented earlier 
(i.e., the writing process, the computers, mini«*lessons , conferencing, and 
monitoring) . 

Information on the course of implementation of these program elements 
was collected through three process evaluation activities, classroom 
observations, teacher logs, and follow-up meetings. Initial information on 
start-*up dates for implementing the program was determined when RBS 
telephoned the principals of the participating schools at the beginning of 
the school year. 

A Classroom Observation Form was developed to collect information 
during observations of participating teacher classrooms. Three rounds of 
observations were conducted, two by RBS (one at the beginning and one at the 
end of the school year) and one by the DPI (mid-year). Teachers' 
self-report data on their implementation of the program elements was 
documented through a Teacher Log and through discussions at follow-up 
meetings. Two follow-up dinner meetings were convened, the first on 
November 28, 1989 and the second on February 15, 1990. The major objectives 



for each meeting were» 1) to facilitate sharing/communication among Project 
participants, 2) to identify any additional assistance and support needed by 
participants to implement the Project, and 3) to develop plans to provide 
the needed assistance in a timely fashion. A third meeting, on May 29, 1990 
was held to facilitate sharing/communicai: ion among Project participants and 
to discuss future plans. 

The outcome evaluation focused on the impact of the Project on student 
writing proficiency, and on student enjoyment of writing and of using the 
computer for writing. In planning for the outcome evaluation, RBS developed 
a document describing the general approach, instrumentation, and scoring. 
As stated previously, the evaluation plan incorporated a pretest-posttest 
control group design. 

Writing Skills. In order to assess student writing skills. Project and 
control teachers administered a Pre-Writing Assessment at the beginning of 
the school year, and a Post-Writing Assessment at the end of the school 
year. The assessments required students to produce a short writing sample 
in response to the following prompts: 

• "Think about one change that you would like to make in your school 
and why you would like to make that change. Give reasons why the 
change should be made, and explain how the change would benefit the 
school. . . " (Pre-Writing Assessment) . 

• "Think about one change that you would like to make in the town or 
city in which you live and why you would like to make that change. 
Give reasons why the change should be made, and explain how the 
change would benefit your town or city..." (Post-Writing 
Assessment) . 

These prompts were similar to those used in the 1985 Delaware Writing 
Assessment Project, and were determined to have the potential for producing 
a range of scorable student responses. They were also judged to be free of 
bias, interesting to students, and unambiguous. 

Enjoyment of Writin g and Computer Learning . Information on student 
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attitudes was collected through a Student Survey attached to each writing 
assessment. Each survey consisted of the same eleven multiple choice Items 
addressing student experiences with writing assignments and computer use* 
Additional impact data were obtained through teacher reports and anecdotes 
shared at follow-up meetings and during school visits and telephone 
contacts. 

Results 

The following section of the report presents the results of the 
data analysis. The findings are discussed in terms of the process and 
outcome evaluation questions. 
Process Evaluation Results 

The evaluation design, as illustrated in Figure 1, specifies four 
process evaluation questions to address the implementation of the Project. 
The results of the process evaluation activities are reported around these 
questions . 

Was adequate orientation and assistance provided for planning and 
selection of hardware /software? Thirteen Project teachers attended the 
two-day orientation session. The Readiness Checklist administered during 
the first day indicated that perceptions of the district/ school leadership's 
commitment and support for the Project, as well as the teachers' own 
commitment toward the Project, were very high. The results also indicated 
that, while teachers had previously attended workshops on the writing 
process, they had only limited experience using a computer in personal 
writing tasks. 

The results of the Inservice Evaluation Form, administered at the 
conclusion of the second day of the orientation workshop, were very 
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positive* In terms of implementation readiness « respondents indicated an 
awareness of the writing process and how to use the computer to teach 
writing for classroom instruction. Although most teachers were exploring 
hardware and software options with administrators , a few indicated that 
either the selections were being made for them or that they were not able to 
contribute to the decisions* Most teachers also indicated a need for 
further training on the hardware, software » and to a lesser extent on 
the writing process* To address these training needs t a week-long workshop 
was scheduled for the end of August* 

RBS*s evaluation of the orientation session for principals and computer 
coordinators indicated the objectives of the session were met. In addition, 
participants exhibited enthusiasm and support for the Project* 

Was the appropriate hardware and software ordered and delivered on 
time? The selections of hardware and software packages for the Project were 
based on the information gained at the orientation sessions, the Resource 
Book developed by RES and the DPI, and through demonstrations and 
discussions with vendors and district administrators. 

The School Hardware /Software Decisions Form, completed by 
administrators in each district in conjunction with the Project teacher, 
indicated that all sixteen schools selected and purchased at least six 
computers, one printer, and writing software by early June. In terms of the 
computer hardware, eight districts selected Apple Macintosh computers (Plus, 
BE, II). five selected Apple lie computers, one district selected Smart 
Micro (IBM compatible), one selected Apple IIGS, and one selected Hyundai. 
The writing software selected and ordered to support implementation of the 
Project included: Writer^s Helper II (seven districts), Success with Writing 
(two districts), and Bank Street Writer III (two districts). Also, two 
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districts selected each of the following word processing software packages: 
MacWrite, Word, Wordbench, WordPerfect , and Works. 

Following the training sessions at the end of August, the Telephone 
Readiness Checklist was administered to all principals In order to Identify 
staff readiness to implement the Project. Only two principals Indicated 
that Project teachers definitely needed additional hardware and software 
training, and all felt that training on the writing process was sufficient. 
In most cases, the hardware and software had been received and set up, and 
only one principal reported that the equipment had not been tested. A few 
principals, however, reported minor problems (e.g., all materials 
software, printers, discs not received, electrical hook-ups and computer 
tables needed, defective monitor). These problems were followed up by RBS 
and the DPI. 

Was adequate inservice training in computer literacy, the writint^ 
process^ and the use of computers in teaching writing provided to Project 
teachers? Nine Project teachers participated in the workshop on the 
hardware and 8<=»lncted software. Feedback on the Software Training 
Evaluation Ff. rr. showed that high ratings were given to both the sessions and 
the presenter''. Overall, teachers felt that they received the support they 
needed on ur,a.ng the hardware and software. In terms of implementation 
readiness, r.hey reported feeling comfortable with using the software for the 
writing and revising stages, but a little less comfortable with using the 
software for prewritlng or planning. Although most teachers felt that they 
needed additional practice before training students, their major concern 
seemed to be with managing a classroom which contained computers. These 
concerns were monitored by the DPI and RBS through school visits and 
personal contacts. 
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Did the teachers start-up on time and Imglement the key program 
elements? Program start-up dates were provided by principals as part of the 
Telephone Readiness Checklist, administered in late August. Four principals 
indicated the start-up date for implementing the program would be the first 
week in September, two principals indicated mid-September, two indicated 
late September, and eight indicated the program would start "as soon as 
possible." The DPI and RBS continued to monitor each teacher's progress in 
beginning implementation. Although most teachers began using their 
computers by October, the initial start-up generally involved several weeks 
of "trouble-shooting" (i.e., setting up the classroom, debugging software, 
managing the classroom, ordering materials, maintaining Teacher Logs). 

Information on program implementation was collected through conducting 
classroom observations, using the Classroom Observation Form, reviewing 
Teacher Legs, and discussing the Project with teachers and students. 
Although the Logs were scanned during observations and requested for 
submission at the end of the school year, only half of the Project teachers 
recorded this information on a regular basis; the remaining teachers 
reported it was difficult to find time to recall and enter the required 
information in the typical "hectic" school day. However, those who did 
maintain the Teacher Log indicated that it provided useful information for 
their instructional planning. 

Over the course of the first round of observations, cc ducted by RBS in 
mid to late October, technical problems were resolved and almost totally 
eliminated. When a school was identified as needing technical support and 
assistance, tnis information was fed back to the DPI and follow-up was 
provided. For example, computer vendors were contacted if hardware was the 
problem, the DPI provided technical assistance to help implement software 
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and address software 'bugs,' and visits among Project staff were 
facilitated. 

In terms of teaching writing, the observations indicated that all 
teachers were focusing their instruction on the writin.g process, i.e., 
planning, revising, and editing techniques, and a few teachers were using a 
mini-lesson format. However, in discussions with teachers, most indicated 
that they did use mini«*lessons on a regular basis, but use of such lessons 
varied with the class and assignment. Also, approximately half of the 
teachers were observed conferencing with individual students. Again, it was 
not expected that all students would be observed to be at this stage in the 
writing process. In several classes students were observed conferencing 
with each other, an activity which some teachers were using prior to or in 
lieu of student**teacher conferencing. This strategy seemed to work well; 
students appeared to enjoy the "author-to-author" interaction with their 
peers and to have the necessary skills to provide helpful feedback. 
Overall, teachers were observed providing extensive individual monitoring 
and assistance to students working on their writing assignments and also to 
students working on computers. 

Observations concerning computer use were also very positive. All 
teachers had developed a system for computer use such that every student was 
scheduled to be on the computer one or more times a week, usually to work on 
and complete at least one assignment. In many classes tht* schedule was 
posted. Teachers had from six to ten computers available (and 1 or 2 
printers), and they were all in use during each observation, by one or two 
students, for an average of 30 minutes. 

Teachers were observed trying out two different instructional 
strategies for incorporating computer use into their English classes. Some 
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had all students working on the same writing ass'.gnment, with some students 
rc«4'^pleting the assignment on the computer. Others had groups of students 
working on different assignments, so that each group had an opportunity to 
complete each assignment on the computer. 

Over the course of the second and third rounds of observations, few 
technical problems were noted. Again, in those few cases, where a printer, 
mouse, or terminal was in need of repair, this information was fed back to 
the DPI and needed support was provided. In all classes, the computeru were 
up and running, and their use appeared to be integrated into the regular 
class schedule. In most cases, students were using the computer for 
drafting (usually copying a hand-written assignment) and editing. These 
observations also indicated that all teachers were focusing their 
instruction on the writing process, and many teachers were using a 
mini-lesson format. However, mini-lessons seemed to be less frequently used 
near the end of the school year. 

During these two rounds of observations, most teachers were observed 
conferencing with individual students, usually for 'discussion* rather than 
•evaluation' purposes. However, a few teachers (i^e., teachers with larger 
classes and/or classes with management problems) reported difficulty in 
finding time to conference with individual students. In some cases, 
teachers were also conducting conferences with small groups of students • 
Again, it was not expected that all students would have reached this stage 
in the writing process. In many cases students were observed working in 
pairs or groups at the computer for both problem solving and sharing ideas 
and writing. In these classes, students seemed to work well together and 
much of this group work seemed to be spontaneous. Overall, teachers 
continued to monitor and assist students working on their writing' 
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assignments. During later observations, the assistance and monitoring of 
stuaents working on computers was greatly reduced and provided on an 
as-needed basis. 

During the three follow-up meetings teachers shared their positive 
experiences and problems wi^-h implementing the program; they offered the DPI 
and RBS recommendations for continuing the Project; and, RBS summarized the 
observation findings and discussed the evaluation and documentation plans. 
Several teachers even shared their students' writing and special class 
writing projects. Teachers informally reported these dinner meetings to be 
very helpful and welcomed the opportunity to meet and share information with 
other Project teachers. 
Outcome Evaluation Results 

Three outcome evaluation questions were specified in the design of the 
study as described earlier. The findings with respect to each of these are 
presented below. 

Were student writing skills enhanced as a result of participation in 
the project? Student writing skills were assessed for experimental and 
control students on a pre- and post-test basis. All pre- and post-writing 
assessments were sent to an independent third party for scoring. A total of 
5,887 papers were scored. The scoring system involved four analytic 
factors: focus » organization, development, and language conventions. The 
focus factor was used to assess the clarity and consistency of the main 
point or subject and its appropriateness to the audience and purpose as 
specified by the prompt. The organization factor dealt with the clarity and 
effectiveness of the plan or arrangement of ideas and the use of 
organizational methods or strategies appropriate to the audience and purpose 
of the prompt. The development factor involved the elaboration of the main 
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point or subject using examples, specific details, and supporting 
information. The conventions factor was used to assess the correct use of 
standard English including grammar, sentence structure spelling, 
punctuation, and capitalization. 

The factors were scored on a six-*point scale according to separate sets 
of criteria and recorded as four independent scores plus an equally weighted 
total score. The six-*point scale ranged as follows} (1) seriously 
deficient, (2) moderately deficient, (3) slightly deficient, (4) moderately 
proficient, (5) proficient, and (6) exceptionally proficient. Specific 
criteria for review of each factor are presented in the appendix. All 
papers were scored by two independent readers, and discrepancies (papers 
with score differences of more than one point) were scored a third time 
(adjudicated) by table leaders. Practice sets and validation packs were 
developed and used for training. Scoring procedures employed initial 
training, practice sample scoring, validation procedures and consistency 
checks. Reader reliability checks were made and reports issued beginning 
with the second day of scoring. Condition codes were recorded in place of 
scores when one of the following response conditions were encountered during 
scoring: (A) blank, (B) illegible, (C) off topic, (D) insufficient to 
score, or (E) predominately in another language. 

A final score was arrived at for each of the four factors, and a total 
score consisting of the sum of the final four factor scores was calculated. 
In obtaining these scores, certain guidelines were followed. If there were 
only two readings, the final score was simply the average of the two reading 
scores. If a third reading was necessary, the final sore was the third 
reading score. If the same condition code was given for the two reading 
scores, the condition code was used as the final score. However, if two 
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different condition codes were given or a numerical code and a condition 
code were given the final score was the third reading score. The total 
score was computed as the sum of the four factor scores* with condition 
codes assigned a value of zero. 

Two approaches were used in the analysis of the data on student writing 
skills. The first involved an analysis of covariance of mean writing 
post**test scores comparing experimental and control group students across 
all schools. Table 1 shows the results of this analysis. The second 
approach analyzed program effects by comparing gains for the experimental 
and the corresponding control group within each school. The results of this 
latter approach are shown in Tables 2 6. In comparing gains for the 
experimental and control group within each school* schools not having a 
control group were evaluated by using the overall control group school mean 
in place of what would have been the school *s control group. 

Table 1 shows results of an analysis of covariance of post-*test writing 
assessment scores using the pre-*test as a covariate. Separate analyses were 
conducted for each score. Results show the total number of subjects 
analyzed (N), the adjusted post-test means for experimental (X) and control 
(C) , the F value (F), and the level of statistical significance obtained 
(Sig.) 

As can be seen in Table 1, the results reflect very favorably on the 
experimental program. Results of comparisons between experimental and 
control {t^roups on organization, development, conventions and total scores 
show statistically significant differences favoring the experimental 
program. Only the focus score failed to show any significant difference. 

An examination of Tables 2-6 shows similarly favorable results at the 
individual school level. Here, in these tables, experimental and control 
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Table 1 



Analysis of Covariance of Post Student Writing Assessment 
Using Pretest as the Covariate 



Score 


N 


Adiusted Posttest Mean 
X U F 


SiQ 


Total 


2285 


16.11 


15.82 


9.34 


.002 


Focus 


2275 


4.18 


4.15 


1.78 


NS 


Organization 


2285 


3.93 


3.83 


13.13 


.001 


Development 


2285 


3.97 


3.90 


4.79 


.02 


Conventions 


2285 


4.03 


3.96 


8.89 


.003 
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Table 2 

Writing Assessment Gains By School 
(Total Score) 



School 


Group 


N 


Pre 


Post 


Gain 


t-siq 


[A] 


X 


84 


15.80 


16.07 


.27 


NS 


C 


158 


16.10 


15.72 


- .38 


.05 


[B] 


X 


47 


15.17 


15.91 


.74 


.01 


c 


65 


16.24 


15.28 


.04 


NS 


[c] 


X 


74 


13.44 


14.47 


1.03 


.or^ 


c 


83 


15.28 


15.87 


.59 




[D] 


X 


81 


14.14 


15.70 


1.56 


.001 


c 


201 


14.85 


16.48 


1.63 


.001 


[E] 


X 


89 


13.54 


17.18 


3.64 


.001 




c 


60 


15.63 


17.08 


1.45 


.001 


[F] 


X 


82 


13.41 


14.79 


1.38 


.001 




c 


81 


13.98 


14.85 


.86 


.001 


[G] 


X 


108 


17.05 


18.01 


.96 


.001 




c 


35 


16.05 


15.58 


- .47 


NS 


[H] 


X 


107 


13.36 


14.74 


1.38 


.001 




c 


83 


15.28 


15.87 


.59 


wt m w» mm 


[I] 


X 


49 


17.71 


18.98 


1.27 


.001 




c 


38 


15.91 


16.20 


.29 


NS 




X 


22 


14.68 


14.41 


- .27 


NS 




c 


83 


15.28 


15.87 


.59 




[K] 


X 


89 


15.00 


14.80 


- .20 






c 


49 


14.95 


16.20 


1.25 


• 01 


[L] 


X 


58 


16.45 


16.89 


44 






c 


97 


15.04 


16.30 


1.26 


.001 


[M] 


X 


67 


16.78 


16.87 


.08 






c 


35 


15.60 


16.11 


.51 


NS 


[N] 


X 


98 


14.96 


15.91 


.95 


.001 




p 


1 7 
1/ 


1 C 1 C 

lb. ID 


15.56 


.41 


NS 


roi 


X 


fi7 






i . J i 


.001 




c 


141 


15.58 


15.73 


.15 


NS 


[P] 


X 


59 


15.09 


17.06 


1.97 


.001 




c 


107 


13.60 


14.29 


.69 


.01 


MEANS 


X 


74 


15.00 


16.03 


1.03 






c 


83 


15.28 


15.87 


.59 





Gr. Gain 



X 
X 
X 

c 

X 
X 
X 
X 



c 
c 
c 
c 

X 



5< 
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Table 3 



Writing Assessment Gains By School 
(Focus Score) 



School 


Group 


N 


Pre 


Post 


Gain 


t-s1a 


Gr. Gain 


LAJ 


A 

c 


158 


& 1 9 

4.19 


4-12 


no 
- .07 


no 
NS 


X 




Y 
A 

C 


65 


4.24 


L 1 1 

4.28 


.04 


NS 


X 


rn 


y 
A 




0 • DO 




99 


.UU3 


X 




c 








.18 






mi 


y 

A 

c 


fil 

oi 

201 


J • /O 

3.90 


4.29 


• JO 

.39 


.UUi 

.001 


c 




y 

A 

c 


OO 

60 


4.05 


A A A 

4.50 


DC 
.00 

.45 


.UUI 
.001 


X 


[F] 


X 

c 


82 
81 


3.55 
3.71 


3.85 
3.91 


.30 
.20 


.001 
.001 


X 




y 

A 

c 


1 HQ 

34 


4,26 


A AC 

4.11 


OA 

.c4 

.15 


nn 1 
.UUi 

NS 


X 




y 
A 

c 


1 n? 




i •oO 


91 

.18 


.UUI 


X 




y 

A 

c 


38 


/I fil 
4.08 


H •oO 

4.11 


9K 

.03 


.UD 

NS 


X 


m 


Y 
A 


99 






no 




c 




C 








.18 








Y 
A 

C 


c5y 
49 


3.89 


4-22 


- ,07 
.33 


NS 
.01 


c 


ri 1 


y 

A 

c 


D/ 

96 


3.95 


4.29 


.05 
.34 


NS 
.001 


c 




y 

A 

c 


0/ 

35 


4.03 


A A A 

4.44 
4.16 


.04 
.13 


hit* 

NS 
NS 


c 


[N] 


X 

c 


97 
17 


3.98 
3.94 


4.09 
4.09 


.11 

.15 


NS 
NS 


c 


[0] 


X 

c 


67 
141 


3.63 
4.07 


3.89 
4.09 


.26 
.02 


.005 
NS 


X 


[P] 


X 

c 


58 
105 


3.98 
3.62 


4.40 
3.74 


.42 
.12 


.001 

NS 


X 


MEANS 


X 

c 


74 
83 






.22 
.18 




10=x 

6=C 
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Table 4 

Writing Assessment Gains By School 
(Organization Score) 



1 School 


Group 


N 


Pre 


Post 


Gain 


t-sig 


[A] 


X 


84 


3.84 


3.98 


.14 


NS 




C 


158 


3.90 


3.81 


- .09 


NS 


1 [B] 


X 


47 


3.59 


3.85 


.26 


.005 




c 


65 


3.86 


3.95 


.09 


NS 


1 [c] 


X 


74 


3.28 


3.49 


.21 


.005 




c 








.17 


■•www 


1 [D] 


X 


81 


3.36 


3.86 


.50 


.001 

• W X 




c 


201 


3.59 


3.97 


.38 


.001 


1 [E] 


X 


89 


3.22 


4.27 


1.05 


.001 

• W X 




c 


60 


3.82 


4.20 


.38 


•001 


1 [F] 


X 


82 


3.26 


3 5Q 




nm 




c 


81 


3.26 


3.58 


.32 


.001 


> [6] 


X 


108 


4.16 


4.41 


.25 


.001 

• vv X 




c 


36 


3.93 


3.79 


.14 


NS 


[H] 


X 


107 


3.18 


3.59 


41 






c 








.17 




[I] 


X 


49 


4.29 


4.60 


.31 


01 

• V/ X 




c 


38 


3.87 


3.88 


.01 


NS 


[J] 


X 


22 


3.56 


3.45 


_ 11 






c 








.17 




[K] 


X 


89 


3.63 


3.53 


• X Vp 






c 


49 


3.64 


3.94 


• 30 


•01 


[L] 


X 


58 


3.97 


4. 14 




• W9 




c 


97 


3.69 


3.97 


• 28 


• 001 


[M] 


X 


67 


4.06 


4. 10 


OA 


no 




c 


35 


3.77 


3.90 


.13 


NS 


[N] 


X 


98 


3.57 


3.88 


.31 


.001 




c 


17 


3.65 


3.74 


.09 


NS 


rm 


Y 
A 


0/ 


3 . 17 


3.62 


.45 


.001 




c 


141 


3.77 


3.79 


.02 


NS 


[P] 


X 


59 


3.64 


4.17 


.53 


.001 




c 


1U7 


3.26 


3.46 


.20 


.005 


MEANS 


X 

c 


74 
83 






.30 
.17 





Gr. Gain 



X 
X 
X 
X 
X 
X 
X 
X 
X 

c 
c 
c 
c 

X 
X 
X 



12=X 
4=C 
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Table 5 

Writing Assessment Gains By School 
(Development Score) 



School 


Group 


N 


Pre 


Post 


Gain 


t-sig 


FA] 


X 


84 


3.86 


3.87 


.01 

• w * 


NS 


c 


158 


3.97 


3.79 


- .18 


.01 


[B] 


X 


47 


3.67 


3.88 


.21 


.05 


c 


65 


4.14 


3.98 


- .16 


NS 


[C] 


X 


74 


3.12 


3.51 


.39 


.001 

• wV A 




c 








.17 






X 


81 


3.42 


3 85 




001 




c 


201 


3.63 


4.13 


• 50 


•001 


TEl 


X 


89 


3 0/ 


4 11 




noi 




c 


60 


3.81 


4-27 


•46 


• 001 


FFl 


X 

A 








• JO 


nni 

• UUi 




c 


81 


3.49 


3.64 


.15 


.05 


[G] 


X 


108 


4.32 


4.52 


20 


005 




c 


36 


4.03 


3.86 


• 17 


NS 


[Hi 


X 


107 


3.15 


3.60 

^ » WW 




noi 




c 








.17 




[I] 


X 


49 


4.43 


4.83 


40 

• "TV/ 


001 




c 


38 


3.82 


4.03 


•21 


• 05 


J 


X 


22 


3 72 


1 4^ 


• 97 
■* •&# 


no 




c 


«• «■ M «• 






• 17 




FKl 


X 


89 


3 80 


Ad 


nd 


Pi J 




c 


49 


3.72 


4.07 


•35 


• 005 


ru 


X 


58 


4 12 




nA 

• UO 


no 




c 


97 


3.75 


4.09 


.34 


• 001 




X 


67 


4 \9 




- •UD 


no 




c 


35 


3.89 


3.97 


.08 


NS 


[N] 


X 


98 


3.64 


3.90 


.26 


.005 




c 


17 


3.76 


3.76 


.00 


NS 


[0] 


X 


6; 


3.23 


3.60 


.37 


.001 




c 


141 


3.82 


3.86 


.04 


NS 


[P] 


X 


59 


3.64 


4.16 


.52 


.001 




c 


107 


3.32 


3.55 


.23 


.005 


MEANS 


X 

c 


74 
83 






.29 
.17 





Gr. Gain 



X 
X 
X 
C 
X 
X 
X 
X 
X 
C 
C 
C 
C 



X 

5=C 
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Table 6 



Writing Assessment Gains By School 
(Conventions Score) 



School 


Group 


N 


Pre 


Post 


Gain 


t-sig 


6r^ Gain 


TAT 


X 
C 


84 
158 


3.99 
4.04 


4.02 
4.01 


.03 
- .03 


NS 
NS 


X 

A 


FBI 


X 

c 


47 
65 


3.86 
3.99 


4.06 
4.06 


.20 
.07 


.01 

NS 


Y 
A 


rci 


X 

c 


74 


3.47 


3.68 


.21 
.14 


.005 


Y 
A 




X 

A 

c 


81 
201 


3 60 
3.72 


3.87 
4.08 


.27 
.36 


.001 
.001 






X 

A 

c 


8Q 
60 


3.66 
3.95 


4.18 
4.11 


.52 
.16 


.001 
.05 


y 
A 


[F] 


X 

c 


82 
81 


3.40 
3.53 


3.73 
3.72 


.33 
.19 


.001 
.005 


X 




X 

c 


108 
35 


4.15 
3.99 


4.42 
3.93 


.27 
- .06 


.001 
NS 


Y 
A 




X 

A 

c 


107 


3.39 


3.68 


.29 
.14 


.001 


Y 
A 


rn 


X 

c 


49 
38 


4.38 
4.14 


4.69 
4.18 


.31 
.04 


.005 
NS 


Y 

A 




X 

c 


22 


3.61 


3.70 


.09 
.14 


NS 






X 

A 

c 


49 


3 80 
3.t^:) 


3.97 


04 
•28 


NS 

no 

•005 


L 


ri 1 


X 

A 

c 


97 


4 . " 
3!"C 


A 1A 
3.97 


.27 


no 

.001 


C 


[Ml 


X 

c 


67 

w f 

35 


4. Is 

3.9^ 


4.25 

I * Urn %J 

4.08 


.06 
.17 


NS 
NS 


L 


[N] 


X 

c 


98 
17 


3.78 
3.79 


4.09 
3.97 


,31 
.18 


• 001 

NS 


X 


[0] 


X 

c 


67 
141 


3.40 
3.92 


3.62 
3.98 


.22 
.06 


• 005 
NS 


X 


[P] 


X 

c 


59 
107 


3.90 
3.47 


4.31 
3.54 


.41 

.07 


• 001 

NS 


X 


MEANS 


X 

c 


74 
83 






.23 
.14 




11=X 
5< 
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groups for each Individual school are shown. For each group, the number of 
subjects (N) with complete pre-* and post^test records is shown. Also shown 
are the mean pre-and post-test score and the gain or difference (Diff) 
score. The results of a correlated t*test (t-sig) comparing pre- and 
post-test scores are also represented in terms of the significance level 
reached* Finally, the group with the greater gain (Gr. Gain) is identified 
in the final column of the tables. For each score type listed in these 
tables, the number of cases where experimental school groups gained more 
than the controls exceeded by a wide margin the number of cases in which the 
controls outgained the experimentals • 

Thus, the weight of evidence indicated by the findings supports the 
contention that the computer writing program does indeed serve to enhance 
the writing skills of participating students. 

Did students get more enjoyment of writing as a result of participation 
in the project? Student enjoyment of writing was assessed using a single 
question on the student attitude survey administered to both experimental 
and control groups on a pre- and post-test basis. The question content and 
analysis results are given below. 



Question: Do you enioy workina 


on wr 


iting assignments? 


1. 


Almost Always 








Often 






3 . 


Sometimes 






4. 


Rarely 






5. 


Never 






Group 


N Pre 


Post 


Diff t-s^ Gr. Gain 


X 


1119 2.90 


2.67 


- -23 .001 X 


C 


1099 3.07 


3.08 


•f .01 NS 


Since the response scale for the 


item is arranged from "1 « Almost 


i^iways" 


to **5 = Never," movement 


from 


a higher pre-test mean to a lower 
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post-*te6t mean is interpreted to be a more favorable response (i«e., a gain 
in enjoyment). As can be seen by the results, only the experimental group 
gained in its enjoyment of writing from the beginning to the end of the 
academic year, and this gain was substantial as well as being statistically 
significant. 0n-*8ite observations and discussions with teachers and 
students at the experimental group sites also confirmed that experimental 
students gained in their enjoyment of the writing process. 

Did students enjoy using the computer for learning and practicing 
writing skills? Student enjoyment of use of the computer for learning and 
practicing writing skills was also assessed using a single item on the 
student survey. The survey question and the results for experimental and 
control groups are presented below. 

Question; Do you enjoy using a computer for writing? 

1. Almost Always 

2. Often 

3. Sometimes 

4. Rarely 

5. Never 

Group N Pre Post Dif f t->sig Gr. Gain 

X 1113 2.39 1.70 .69 ♦001 X 

C 1095 2.78 2.66 - .12 .01 

The response scale ranges from a "1 « Almost Always" to a "5 = Never", 
and thus changes in the direction of more enjoyment from pre- to post-test 
would result in a negative difference score. The magnitude of the negative 
ditferenve scnre would indicate the extent of the more favorable change 
experienced. 

As can be seen by the above results, the experimental program group 
shows a gain in enjoyment of computer use for learning and practicing 
writing skills. The gain is also more than five times that of the control 
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group. This is to be expected since the exposure of the control group 
students to computers for use in writing on a formal basis was to have been 
of a limited nature. 

It is clear, then, that the experimental program group enjoyed using 
the computers. On-site observations and discussions with teachers and 
students served to confirm this finding as well. 



Conclusions and Implication s 
Based on the first-year evaluation study of the computer writing 
program, several conclusions can be drawn. They are as follows: 

1. In general, adequate orientation and training were provided to the 
participating schools as well as help in selecting and acquiring 
computer hardware and software. 

2. Most program sites started on time and implemented the program as 
planned. 

3. Despite all efforts, a few of the schools did not take advantage of 
the special training opportunities and technical assistance 
services and, in some cases, struggled somewhat during 
implementation as a consequence. 

4. On-site visitation and monitoring of the first year of 
implementation of a computer writing program is essential to its 
success . 



5. Student writing skills are enhanced through use of computers 
within the context of a process approach to writing. 

« 

6. Students enjoy writing more when using computers in conjunction 
with their normal writing instruction program. 

7. Students enjoy using the computer for learning ana practicing their 
writing skills. 

8. Teachers enjoy using the computer for writing instruction. 

This study has provided an opportunity to assess the effectiveness of a 
program designed to introduce computers into the English/Language Arts 
classroom to assist in the teaching of the writing process. The findings 
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are quite favorable to the program, especially in light of the fact that 
this was its first year of implementation. Despite these initially positive 
findings, several questions remain for further study: 

• What is the maximum potential level of effectiveness of the program 
once implementation problems have been overcome and program 
operations have stabilized? 

• What variables mitigate against successful j^mplementation and 
stabilization of the program? 

• In what ways do teachers alter their instructional styles to 
accommodate the use of computers in the classroom? What seems to 
work best? What doesn't? 

• In what ways do students alter their learning styles to accommodate 
the use of computers for learning in the classroom? What seems to 
work best? What doesn't? 

The present time represents a critical period for continued study of 
the computer writing program. It is a window of opportunity which may not 
be open very long. During the current year of study, it was possible to 
employ a stringent experimental design by obtaining comparable control 
groups of students that could be expected to have limited exposure to 
computer-assisted instruction. As schools purchase additional computers, 
this will become less feasible and will increase threats to internal 
validity in the evaluative research design. It is recommended, therefore, 
that if continued study is to be undertaken on this program, that it be 
carried out during the upcoming school year, so that more definitive results 
can be gathered over a two-year period of program operation. The two-year 
findings could then serve as a relatively firm foundation for state- and 
local-level policy decisions regarding the future course of 
computer-assisted instruction in Delaware. 




21 



o7 



Appendix 



IFRir 



22 



8 



APPENDIX I PROJECT DOCUMENTS AND DATA COLLECTION INSTRUMENTS 
Memorandum of Agreement 

Schedule of Activities for Delaware Middle School Writing Project 

Guidelines for Selection of Control or Comparison Group 

Agenda for April 17, 1989 and May 15, 1989 Orientation Sessions 

Readiness Checklist for Participation in the Middle School Computer-Assisted 
Writing Project 

Teacher Inservice Evaluation Form 
School Hardware /Software Decisions Form 
Telephone Readiness Checklist 
Software Training Evaluation Form 
Classroom Observation Form 
Teacher Log 

Writing Assessment and Scoring Procedures 

Pre-Writing Assessment, Student Survey #1, Instructions j Post-Writing 
Assessment, Student Survey #2, Instructions 

Using Computer Technology in Support of Student Writing: A Resource Book 



MEMORANDUM OF AGREEMENT 



TO: 



SUBJECT: 



Mr. Sidney B. Collison 
Deputy State Superintendent 
Instructional Services Branch 

MIDDLE LEVEL EDUCATION PROGRAM FOR WRITING 



The 



School District hereby agrees to 



participate in the project to improve the writing skills of students In 
all of the following ways: 

1. An English teacher will be named by the district to staff the 
designated middle school (grades 7 or 8) classroom. 

2. The English teacher will be made available for the scheduled 
training (two days) In the use of computers and related software. 

3. At least six computers, one prlntV^ selected software and supplies 
will be purchased and Installed in the designated classroom. 

4. The computers will be housed in a secure area and will be maintained 
in order to keep them operative. 

5. The computers will be used exclusively for teaching the writing 
process and for related word processing activities. 

6. Representatives from the Department of Public Instruction and from 
Research for Better Schools will be permitted to visit the 
designated classroom and to work with the assigned teacher In the 
implementation and evaluation of the program. 

7. The district will be responsible for the expenses associated with 
the preparation of the designated classroom I.j order to achieve the 
appropriate computer configuration. 



On this 



Signed 



day of 



Superintendent 
In the year 1989. 



Return to: Dr. Gary L. Houpt 
State Supervisor 
English Language Arts 
Department of Public Instruction 
Townsend Building 
P. 0. Box 1402 
Dover, DE 19903 
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Schedule of Activities for Delaware Middle School Writing Project 



1989 1990 

Training of Participants MAMJJASONDJFMAMJJA 

• Orientation for teachers • • 

• Orientation for administrators • 

• Software training sessions • • • • 

• Writing process course • • 

Selection of Hardware and Software • • 



Implementation of Writing Instruction 
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Evaluation of Project 

• Identify control group • 

• Collect writing samples • • 

• Observe writing classes ••••••••• 

• Analyze data • • 

• Prepare evaluation report • 



Gulcelines for Selecting a Control or Craparlson Group 



An appropriate Con\:rol Group should be as similar as possible to the 
group participating in the Computer Writing Project. This would include 
variables such as type of students, demographic variables (Grade, Age, Sex, 
Race), and achievement level* In the ideal situation, the only difference 
between the two groups would be that the Project students use the computer 
for writing. The most appropriate Control Groups for the Middle School 
Writing Project are: 

• other classes in the same grade taught by the Project teacher, but 
not using the computer for writing 

• other classes in the same grade at the same school, not taught by the 
Project teacher, and not using the computer for writing 

• other classes in the same grade at a different school, not using the 
computer for writing. 



WORKSHOP FOR DELAWARE MIDDLE SCHOOL WRITING PROJECT 



Date : April 17, 1989 



Presenters : Gary Houpt, DPI 

Ronald Houston, RBS 

Ru88 Dusewlcz, RBS 

Colleen Woznlak, DPI 

Carol Kopay, Milford School District 

Joanne Silvestrl, Apple Computers 

Christine Olson, IBM 



Materials and Handouts : 

• Videotape - The Writing Center, Central Hower High School, 
Akron Public Schools - February 1987 

• Resource Book: Using Computer Technology in Support of 
Student Writing 



Objectives ; 

Participants will be able toj 

• describe the Project purpose, plans, participant 
requirements and documentation procedure; 

• list findings from research studies that support the use 
of computers to assist writing instruction; 

• describe how the computer may assist writing instruction; 

• describe a model program using computers to assist writing 
instruction; and 

• identify several computers listing attributes that support 
their use in the writing classroom. 



Activities : 

9i00-9j45 Presentation of Project : The presenter will describe 

the project purpose, plans and school requirements. A second 
presenter will describe two research articles that support the 
use of computers for writing instruction. 

9:45-10:15 Discussion of Participant Experiences aad Expectations : 

The participants will describe experiences using the computer 
and their expectations for the project. 
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10:15-10i45 D iscussion of Documentation Plan : The presenter will 

administer a checklist of project requirements, present 
documentation plans and begin the documentation activity* 



10j45-10j55 BREAK 



10i55-.llt40 Presentation of Model WritinR Classroom s This activity will 
be conducted in two parts s 1. The presenter will show a 
videotape, prepared by Apple Computers, that presents a 
classroom of students who are learning to write using the 
computer; 2. The presenter (classroom teacher) will describe, 
using a computer, her experiences teaching writing to middle 
school students* 

11:40-12:00 Overview of Participant Inservice or Staff Development 

Activities : The presenter will describe activities designed to 
prepare project participants to implement the computer 
assisted writing instruction. A resource document which will 
be the primary training material will also be described. 



12:00-1:00 LUNCH 



Ii00-3i00 Hardware Demonstration ? Presenters (IBM & Apple) will 

demonstrate computers describing how they may lend themselves 
to assist writing instruction in the middle school classroom. 



Diecuseiont 



3:00-4:00 



Participants will be given an opportunity to ask questions, 
make comments and recommend changes in the project. 



Workshop 



Delaware Middle School Writing Project 



May 15, 1989 



Overview of activities Gary Houpt 

Review of project timeline and 

evaluation Ron Houston 

Russ Ousewicz 

Presentation - Using Computer Technology 

in Support of Student Writing Gary Houpt 

Presentation - Techniques for managing 

computers in the classroom Patricia Masten 



Presentation - Overview of wr '.ing 

software Ron Houston 

LUNCH 

Presentation - Wasatch writing program Adrian Binns 

Maureen Kerr 

Software review (hands on activity) Christine Olson 

Joanne Silvestri 



Questions 
Announcements 



Gary Houpt 
Gary Houpt 
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READINESS CHECKLIST FOR PARTICIPATION IN THE 
MIDDLE SCHOOL COMPUTER-ASSISTED WRITING PROJECT 

SUMMARY OF RESULTS (N«16) 



lean 



Strongly 
Agree 



Strongly 
Disagree 



[4.6 



1. District/school leadership are committed to 5 4 3 2 1 

improving the performance of their middle 
school writing programs. 

1.4.3 2. District/school leadership believe that computer 5 4 3 2 1 
technology may help teachers improve student 
writing. 

3. The middle school language arts teacher selected to participate in the 
writing project: 

• is committed to teaching writing as a process 5 4 3 2 1 

• has participated in a writing workshop that 5 4 3 2 1 
involved him/her in the writing process and in 
considering the implications of that experience 
for writing instruction 

• uses a computer in personal writing tasks. 5 4 3 2 1 

4. District/ school leadership will provide the teacher with sufficient time 
to: 



2.3 



4.1 



4.1 



4.1 



• preview and select software that may help 
students with the writing process 

• develop plans for how to prepare students 
to use the selected software 

• develop with school leadership plans for how 
students will get the opportunity to use 
computers both during and outside of class time. 
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Strongly Strongly 
Agree Disagree 

5 4 3 2 1 



The teacher, with the assistance of district/school leadership, vill 
develop the kind of project plan that satisfactorily sets forth: 

• their project objectives 5 4 3 2 1 

• the activities that teachers and students vill 5 4 3 2 1 
engage in to achieve those objectives 

• a schedule for those activities. 5 4 3 2 1 

District/school leadership and participating teachers are totally 
committed to providing all the necessary information for the project 
evaluation. More specifically, this includes: 

• maintaining teacher and student logs 

• completing periodic teacher and student 
questionnaires 

• maintaining folders of student writing 

• establishing a control group. 



5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 

5 4 3 2 1 



Teacher Inservice EvaXuatioa Form 



SUMMARY OF RESULTS (N«13) 



Strongly Strongly 

Mean Agree Dijiagree 

Project Orientation 

A. 5 I. The objectives of the Writing Project 5 4 3 2 1 

and the Project timeline are clear* 

4.5 2. My role and responsibilities in the 5 4 3 2 1 

Project are clear. 

4.7 3/ There was adequate time in each session 5 4 3 2 1 

for discussion and questions* 

Implementation Readiness 
4.6' 4. 'I am aware of a framework which describes 5 4 3 2 1 
writing as a four-stage process. 

4*7 5. I am aware of how the computer may assist 5 4 3 2 1 
students working through this writing 
process . 

4.2 6. I have an understanding of how the 5 4 3 2 1 
computer will be integrated into my 
classroom instruction. 

4.5 7. I am exploring, with administrators in my 5 4 3 2 1 
district, the hardware and software 
options available to me. 

m 

3.3 8. I feel able to contribute to the decision 5 4 3 2 1 
of selecting the hardware for the Project. 

3.5 9. I feel able to contribute to the decision 5 4 3 2 1 
of selecting the software for the Project* 

_ 4*3 10. The timeline for Project activities 5 4 3 2 1 

(training, hardware/ software selection, 
^ implementation) is feasible. 

~ Training Readiness 

4.0 11. I will need further training on the 5 4 3 2 1 

hardware . 

4.1 12. I will need further training on the 5 4 3 2 1 
■ software. 

2.9 13. I will need further training on the 5 4 3 2 1 

writing process. 

Additional Concerns/SuRgestions (use back of page for response): 
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DELAWARE MIDDLE SCHOOL VRITING PROJECT 
School Hardware /Software Decisions 

**Plea8e complete the following and return to Gary Houpt on or before June 1 
at the following address t Delaware Department of Public Instruction, 
Townsend Buildings P.O. Box 1402, Dover, DE 19901. 

Administrator! 

Teacher : 

School : 

Phone :( ) Date;^ 



1. Our school has decided to purchase the following computer hardware 
to support our implementation of the Writing Project: 
Number : Hardware Type (Model Name) : Manufacturer : 



2. Our school has decided to purchase the following computer software 
to support our implementation of the Writing Project: 

Number : Software Package Name Software Developer : 



3. Will the hardware and software purchased for this project be networked 
at this time? 

Yes ^No 

4. On what aspects of the project does the teacher need training, and 
what will be the source of training to meet these needs? 

Training Needed Source of Training 

Yes No DPI District Both Neither 

a. computer hardware 

b. computer software 

c. writing process 



5. Will the teacher be available to participate in the writing process 
inservice ( ''Teachers Teaching Writing scheduled for August 21-25? 

Yes No 

If no, will the teacher be attending any other writing process inservice 
sessions? 

Yes No 



I 
I 

I 

I 



Principal: 
School: 



Project Teachers 



DELAWARE MIDDLE SCHOOL WRITING PROJECT 
Telephone Readiness Checklist (2) 



District: 



Date; 



Training 



Yes 



No 



Other 



1. Did the Project teacher receive enough 

training on using the computer hardware? 
hardware selected 

2. Did he/she receive enough training on 

using the computer software? 

software selected__ 

3. Did he/she receive enough training on 

teaching the writing process? 

Implementation Readiness 

A. Did you receive all of the hardware 

ordered for the Project? 

5. Is the hardware set-up, tested, and 

ready to use? 

6. Did you receive all of the software 

ordered for the Project? 

&. Is the software set-up, tested, and 

ready to use? 

Try out 

9. Has the Project teacher had the 

opportunity to personally tryout 

the hardware? 

10. Has he/she had the opportunity to 

personally tryout the software? 

Comparison Group 

11. Has a comparison (control) group 

been selected? 

If yes, type; 
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I 

I 

I 

I 



Program Plan Yes No Other 

12. Is there a "program plan" for imple- 

meriting the Project? 

If yes, does the plan address: 

a. classroom management? 

b. time on writing instruction? 

c. time using computers? 

d. how computers will be used? 

e. how to insure equal computer 

use for all students? 

Program Start-Up 

13. What is the start-up date for imple- 
menting the program? 



14. When will the pre-writing assessment 
be administered to the Project 
class (es) ? 



15. When will the pre-writing assessment 
be administered to the comparison 
(control) group(s)? 



Comment s/ Concerns 
16. 
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I 

I 

I 



Mean 
4.2 

A. 3 

A. 6 

A. 6 

A.O 



A.O 
A. A 
A. A 



3.9 
A. A 
A. 2 

A. 3 



3.8 



A.l 



3. A 
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DELAWARE MIDDLE SCHOOL WRITING PROJECT 
Software Training Evaluation Fom* 

Date t 



School I 



Districti 



Software Package(8} Namet 



Directions > The following items relate to the hardware and software package (s) 
you selected and were trained on for the Delaware Middle School Writing Project. 
Please circle the number that most closely matches your response. 



Strongly 
Agree 

Software Training 

1. There was adequate time for practice in 5 4 3 
using the software « 

2. There was adequate time for discussion 5 4 3 
and questions. 

3. The presenter was knowledgeable and 5 4 3 
interesting. 

4. The sessions provided me with the support 5 4 3 
I needed on using the hardware /software. 

5. The capability and features of the soft- 5 4 3 
program(s} were clearly presented. 

Implementation Readiness 

I understand how the software program(s} 

can be used to help students with: 

6. the prewriting or planning stage 5 4 3 

7. the writing or drafting stage 5 4 3 

8. the revising or rewriting stage 5 4 3 

I am able to use the main features of the 
software program(s} that relate to: 

9. the prewriting or planning stage 5 4 3 

10. the writing or drafting stage 5 4 3 

11. the revising or rewriting stage 5 4 3 

12. I feel that I selected the best available 5 4 3 
software package(s) for this Project. 

13. I am ready to train students in using 5 4 3 
this software. 

14. I understand how to integrate this soft« 5 4 3 
ware into my lesson plans for teaching 

writing. 

15. I understand how to manage a classroom 5 4 3 
environment which has computers. 

Additional Concerns/Suggestions (use back of page for response) 



2 
2 
2 



2 
2 
2 



Strongly 
Disagree 



1 
1 
1 



1 
1 
1 



not applicable because 
received no training 



N/A 
N/A 
N/A 



N/A 
N/A 
N/A 
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DELAWARE MIDDLE SCHOOL VRITXNG PROJECT 
Classroom Observation Form 



District: School t 

Teacher; Grade? Subject: 

Observer: Datp: Time: to 



Please indicate vhether or not you observe the followng : 

Observed 

a/ the TEACHING OP WRITING AS A PROCESS 

Pre-Writing 

1. Selecting and limiting a subject. 

2. Developing preliminary outline (generating ideas). 

3. Developing a thesis statement. 

4. Developing a complete outline (organizing). 

Drafting 

5. Selecting an opening statement. 

6. Writing a first draft. 

Revising 

7. Revising drafts. 

8. Preparing final copy. 

9. Other 



B. MINI-LESSONS 

Pre-Vriting 

1. Selecting and limiting a subject. 

2. Developing preliminary outline (generating ideas) 

3. Developing a thesis statement. 

4. Developing a complete outline (organizing). 

Drafting 

5. Selecting an opening statement. 

6. Writing a first draft. 

Revising 

7. Revising drafts. 

8. Preparing final copy. 

9. Other 



C. COMPUTER USE 



1. Adequate schedule posted. 

2. Time per student per week, 
(posted/from teacher) 

3. Number of computers available. 

4. Number of different students observed 
using computers. 

5. Number of computers observed not in use. 

6. Average time per student on computer. 



minutes 
computers 

students 

computers 

minutes 
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Observed 



' D. STUDENT CONFERENCING 



No* of Students Time 

1. Small Grou^ Per Group • ( Total ) Per Group ( Total ) 

No. of Groups — Discussion 

No. of Groups Evaluation — 



2. One -on- One 

No', of Students — Discussion 
No. of Students Evaluation 



Time 

Per Student (Total) 



To a Great Not At 

, Extent All 



E. MONITORING 

1, To what extent did the 
teacher provide individual 
monitoring and assistance 
to students working on the 
compute r? 

2. To what extent did the 
teacher provide individual 
monitoring and assistance 
to students working on 
writing assiRnments ? 



F. ADDITIONAL OBSERVATIONS /COMMENTS 



DELAWARE MIDDLE SCHOOL WRITING PROJECT 
Teacher Log 

District: Schoolt 



Teacher: Grade: Class: 



Directions: Please keep a separate Log for each of your writing classes for 
the 1989-1990 school year. Following each writing period, enter four types 
of information on the Log, as follows: 



1) enter the date, 

2) check off any mini-lesson topics presented that day (write in the 
topic if it is not listed on the form) , 

3) if conferencing occurred that day, indicate the approximate number 
of groups participating in small-group conferencing and the approxi- 
mate number of individual students participating in one -on- one 
(individual) conferencing, and 

4) in the "monitoring" column, for that day, indicate th^ approximate 
number of students using computers and the approximate number of 
these students using computers whose work you monitored (i.e., 
checked and assisted) . 



1-5 [-1 \ 



DELAWARE MIDDLE SCHOOL WRITING PROJECT - TEACHER LOG 



DATE 


1. Selecting & limiting 
a subject 


2. Developing preliminary 
outline (generating ideas). 


3. Developing tfiesis 
statement 


4. Developing complete 
outline (organizing). 


5. Selecting opening 
statement 


2 

6. Writing first draft. r- 


o 

7. Revising drafts. g 


8. Preparing final copy. 


9. Other 


• 






SMALL GROUP C 
(No. of groups) ^^ 


INDIVIDUAL f 
(No. Of students) z 


. — , 

USING COMPUTERS . ^ 


NO. OF STUDENTS MONfTORED | 
USING COMPUTERS C 

1 
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DELAWARE COMPUTER WRITING PROJECT 
WRITING ASSESSMENT AND SCORING PROCEDURES 



RESEARCH FOR BETTER SCHOOLS 
444 North Third Street 
Philadelphia, PA 19123 
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GENERAL APPROACH 



The Delaware Computer Writing Project is attempting to improve the writing 
skills of Delaware Middle School students through use of computer assisted 
instruction in the teaching of the writing process. While this project is only 
in its first year of implementation at a middle school in every district within 
Delaware, its ultimate effectiveness lies with its longer-term impact upon the 
writing skills of the students participating in the program. It is for this 
reason that the principal evaluative tool for assessing the effectiveness of the 
project consists of an assessment of student writing skills. 

For this purpose, a pre- and post- test of writing skills for students 
participating in the computer writing program (the program group) as well as 
students participating in comparison classes (the control group) is being 
conducted. An analysis of gains achieved by the program group relevant to the 
comparison group will indicate the extent of the effectiveness of the computer 
writing program during its first year of implementation. 

INSTRUMENTATION 

Instrumentation used for assessing writing, depends directly upon the type 
of philosophy one chooses to favor with respect to the assessment of writing 
proficiency. Two primary approaches are considered feasible for such an 
assessment: The direct method and the indirect method. The direct method 
involves collecting actual samples of student writing and making judgements 
about proficiency. The indirect method utilizes objective tests to infer 
writing proficiency. While past research has shown relatively strong 
correlations between these different types of assessments, there are distinct 
practical advantages and disadvantages associated with each. A comparison of 
the advantages and disadvantages of both types of assessment is presented below. 
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Direct Assessment 

Advantages 

• Higher fidelity and face validity 
cf the exercise and response. 

• More extensive information provided on 
writing proficiency* 

• Higher relevance to real world writing, 
tasks. 

• Lower test development costs* 

Indirect Assessment 

Advantages 

• Higher reliability of scores. 

• Higher control over the skills 
tested* 

• Lower test scoring costs. 



Disadvantages 

• Higher scoring costs. 

• Lower uniformity 
amongst students on 
proficiencies. 



Disadvantages 

• Lower face validity. 

• Lower relevance to 
real world writing 
tasks* 

• Higher emphasis on 
student reading 
versus writing 
proficiency. 



Based on time and cost constraints, recent trends in approaches to writing 
assessment, and strong preferences toward relevancy of the assessment, the 
direct approach to assessing writing proficiency was selected for use in 
evaluating the computer writing project. 

Having decided on the general approach to the writing proficiency 
assessment, the next decision, concerned the type of writing to be assessed. 
This could include: Description, narration, exposition, or reporting. 
Different exercises and requirements involved in writing an editorial, business 
letter, personal letter, or an analysis of some event could be called for in the 
assessment. After careful consideration of all the options, the decision was 
made to utiliie a single topic or prompt. This prompt was to limit student 
dependence upon actual facts to write a response. 

It v.as also to involve a topic interesting and familiar enough to the 
student to motivate a full and interesting response. In addition, because of 
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the limited amount of time (too short to allow for pre- testing of the topic or 
prompt), a prompt similar to one already tested in a prior statewide 
administration was utilized for the pre- test, with a variant of this prompt for 
post test assessment. 

SCORING 

There are a number of different options t^. could be used for scoring of 
actual writing samples. Each method, in turn, ^ as its own advantages and 
disadvantages. A relative comparison of the advantages and disadvantages of 
five major scoring methods is given in the table on the following page. These 
methods are: holistic, analytical, primary trait, writing mechanics, and T*unit 
analysis. 

Because of the relatively large sample involved in the statewide assessment 
of the computer writing project, the primary methods considered for scoring of 
this assessment were the holistic and a modified analytical. 

Analytical scoring examines individual traits considered important to any 
piece of writing in any context. The traits under consideration for present 
purposes include the following: 

• Organization - response should have a focus and 
be carefully organized with a beginning, middle, 
and end. In addition, there should be a clear 
plan or strategy that is controlled. Although 
there may be a minor lapse or two, the response 
should progress logically from the opening 
statement to the closing statement. 

• Supporting detail- the response should fully 
accomplish the purpose by providing relevant 
reasons and explanations, as requested in 
the composition topic. In addition, some of 
the details must be elaborated or most of 
them extended. 

Mechanics /Usage- the writer should demonstrate 
understanding of the rules of standard 
edited American English. In other words, the 
rules for sentence formation, punctuation, 
capitalization, ?T?eiling, and word choice 
should be consict-^ iwiy applied. In addition, 
the language should oe generally varied. 
There may be minor errors in some of these 
papers. 
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HoliS'iic scoring Involves quickly reading a paper for an overview or 
"whole" Impression and balancing all features rather than addressing specific 
traits. The scoring Is specific to the writing samples being evaluated* A four, 
six, or eight point scale will be developed based upon the range of 
effectiveness reflected in the set of writing samples. Student papers typical 
of each of the score levels (1*6., anchor papers, benchmark papers, range 
finders) will be identified and used as models in assigning scores. As with all 
method:r of r&L .ng writing samples, raters must be carefully trained. Raters 
must be expur.\e:4':ed in language arts, and practiced in grading student papers at 
the middle scS-^w,! level. Initial training for raters will consist of 
approximately a :*.\lf day with follow-up sessions during the course of scoring. 

The firs*'; ^tep in scoring occurs prior to actual scoring. At this point a 
group of th 'Uit experienced raters review a sub-set of papers to identify 
"range-findeu. * , Range-finders are papers that are representative of all the 
papers at .s-^n scoring level. For a four-point scoring scale, there would be 
four rar .line^^ri, ox;e at each level. For an eight-point scoring scale, there 
would lilc^Ase be elghr range finrt^-r??. The range finders would be used in 
training and later as models to assist in scaring. There will be two range 
finders located for each level of scoring. Past experience with the holistic 
approach has indicated that it quickly produce^ consistency among raters. 
Inter-rater reliability generally has been fcur'i to run between .60 and .80 
across a number of studies. All papers will be read by two raters. Past 
research has shown that increasing the number of raters beyond two does not 
appear to appreciably enhance reliability. Differences between the raters by 
more than a point will be resolved by an adjudicat.'.on procedure involving a 
third reader. It can be expected on the basis of past experience with holistic 
scoring that an average of thirty to forty papers can be read by an experienced 
rater each hour, and that six hours of scoring a day should be considered the 
maximum workload. 

To operationalize the scoring for purposes of the Delaware Computer Writing 
Project, all pre- and post-writing samples will be examined for comploto pie-- 
and post-test pairs. Only those with complete pre and post test pairs will be 
approved for scoring. Once the sample has been reduced to complete pre and 
post-test pairs, all writing samples for program and comparison groups will be 
coded on both cover sheets and composition sheets with the same numerical code. 
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This numerical code will distinguish pre-test from post-test, but will be 
embedded in a larger numerical code which will provide randomly generated 
numbers to make pre-tests indistinguishable from post-tests to the raters. 
Thus, raters will not be able to tell pre-tests from post tests during their 
reading and rating of the writing samples. 

Demographic information on the program group will be analyzed and 
descriptive statistics computed based on the cover sheets. Sampling of the 
comparison group will be accomplished by randomly casting out subjects in order 
to achieve a relative match between the comparison and the program group on the 
significant demographic variables. The resulting sample will be subjected to 
rating by a trained cadres of raters. The scoring will be done in May or June 
of 1990. 

Raters will be drawn from a list compiled a few years ago by the DPI. The 
persons on this list have already been trained and utilized in connection with a 
previous writing sample. The most consistent and experienced of those raters 
will be enlisted to take part in the current scoring effort. They will receive 
a half day of training followed by scoring sessions at a time and place to be 
specified at a later date. Procedures for assigning papers for scoring, for 
adjudicating differences in scores, and for reassigning papers will also need to 
be developed. Once scoring is completed, scores will be recorded for each of 
the I.D. numbers identifying the pre-* and post-test for each student. These 
will be coded and entered into the RBS computer system where they will be 
matched with the demographic information already on file. Comparisons will then 
be made between the results of the program and control groups within and across 
districts in order to infer program effects attributable to the Computer Writing 
Project . 

PROJECTED COSTS 

Any projection of costs for scoring and associated training is 
understandably only an estimate at this point in time. Given that, the 
following is the best estimate for the costs of scoring that will be incurred in 
May or June of 1990. 
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Approximately 4,000 pre-tests were sent out to schools in early September. 
Of these 4,000 pre-tests it is estimated that approximately 2,000 were program 
students and 2,000 were comparison students. If we assume, for one reason or 
another, that complete pre-and post-test scores will be obtainable for the total 
program group and the total comparison group of students, then the total pre- 
test sample and post-test sample will equal 4,000 each for a total of 8,000 
writing samples to be scored. 

If we begin with a total of 8,000 writing samples, and multiply that figure 
by two readings each, this yields a total of 16,000 readings. If we then 
multiply this figure by 1.2, supplementing this figure by the estimated number 
of third readings needed in cases where adjudication is required, then the total 
yields 19,200 total readings . required for scoring. If we then divide this total 
by 30 readings per hour, it gives us a total of 640 hours required. Thus a 
total of 640 hours is needed for scoring the 19,200 writing samples. 

If we assume that we will also need three chairpersons or table leaders, 
that will work at a pace of 4 hours of reading per day plus two hours of 
anchoring, then the number of person-days required for scoring can be computed. 
This figure will be 80 person-days for readers plus the time required for the 
table leaders. If we further assume that the scoring is to be done in a one- 
week period, including a day of training, then we will need 80 / 4 « 20 readers 
plus 3 table leaders for scoring, or 23 persons totals for the one-week period. 
If we compute the cost of a person day at an average of $110 per person (based 
on substitute costs) x 23 persons x 5 days, then the total cost of scoring would 
be about $12,650 plus travel, subsistence, and lodging if necessary. Additional 
costs may be associated with training and support during the scoring if outside 
assistance other than DPI is needed. 

These cost estimates represent our best projection at the present time 
given the above sample sizes, without pilot testing the reading and scoring 
process with actual writing samples, and without knowing who might be available 
from the prior list of experienced raters DPI used in the past. Refinements in 
these cost estimates will occur as further progress is made in planning for the 
scoring session. 
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PELAVARE MIDDLE SCHOOL VRZTZNO PROJECT 
PRE-VRZTING ASSESSMENT 



Student Names 



Grade t 



Age I 



Ethnic Group I (check one) 



School Namely 
Teacher; 



Asian 



iOiite 



Dates 



Sext (check one) Male 



Female 



Black 

^ pother 

Districts 



JHispanic 



DIRECTIONS FOR STUDENTS 

Your assignment during this class is to write a composition. The topic for 
the composition is on the next page (page 2). Space is provided to plan or 
outline ideas for your composition. Your plan will not be scored. 

When you finish your plan* read It over and then write your composition on 
the lined paper attached to these directions. You may use handwriting or 
printing when you write your composition. Be sure to write clearly so your'work 
can be read easily. Only the composition on the lined paper will be scored. 

On pages 5 and 6 you will find a student survey. When you finish working on 
the composition, mark your answers to the questions on the student survey. 

Now turn the page and begin. 
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COMPOSITION TOPIC 1 

Think .bout ONE chsngt that you would llkt to make in your ichool and jhy you 
would lik« to maka that changa. Wtitt a compoaition diicutting that changa. 
Give reasons why the change should be made, and explain how the change would 
benefit the school. Remember that you are to write about only ONE change. 

(You may use the space below to plan or outline ideas for your composition. 
The plan or outline will not be scored.) 
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COMPOSITION 1 

(Write your final copy of the composition on the lines below.} 



IF YOU NEED MORE SPACE, CONTINUE ON THE NEXT PAGE. 
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STUDENT SURVEY #1 



Name t ^ n 

School! . Teacher; 

Directions I The questions below are about your writing experiences. Read each 
question carefully and then fill in the circle next to the best answer. Remember 
to fill in only one circle for each question, 

1. When I write something for school, I think the way I write is: 

0 Very Good 
0 Good 
0 Fair 
o Poor 
0 Very Poor 

2. When I write something for school, teachers usually grade the way I write as: 

0 Very Good 
0 Good 
0 Fair 
0 Poor 
0 Very Poor 

3. How often do you use a computer at home for writing? 

0 At Least Once a Day 

0 At Least Once a Week 

0 About Once a Month 

0 Only Once or Twice a Year 

0 Never 

k. Last year, how often did you use a computer at school for \/riting? 
0 At Least Once a Day 
0 At Least Once a Week 
0 About Once a Month 
0 Only Once or Twice a Year 
0 Never 

5. How would you rate your typing or keyboarding ability? 

0 Very Good 
0 Good 
0 Fair 
0 Poor 
0 Very Poor 

6. When you have a writing assignment, how often do you plan what you're going 
to write and how you're going to write it? 

0 Almost* Always 

0 Often 

0 Sometimes 

0 Rarely 

0 Almost Never 

PLEASE CONTINUE ON NEXT PA6E««« 
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7. When a writing assignment is returned, do teachers require you to work on 
the paper again to improve it? 

o Almost Always 

o Often 

o Sometimes 

o Rarely 

o Almost Never 

8. Do you enjoy working on writing assignments? 

o Almost Always 

o Often 

o Sometimes 

o Rarely 

o Almost Never 

9. How often are you required to write a paragraph or more in your school 
assignments? 

o At Least Once a Da/ 

o At Least Once a Week 

o About Once a Month 

0 Only Once or Twice a Year 

o Never 

10. Do you enjoy using a computer for writing? 

o Almost Always 
o Often 
o Sometimes 
o Rarely 

o I Haven* t Used a Computer for Writing 

11. Do you have a computer at home? 

o Yes 
o No 



Thank you« 
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DELAWARE MIDDLE SCHOOL VRXTZN6 PROJECT 
PRE-VRITIN6 ASSESSMENT 
Instructions 

Enclosed are copies of the Pre-Wrltlng Assesment to be administered to 
all students participating in the Computer Writing Project and to students 
In the' Comparison Groups (In most cases, the comparison groups will Include 
all other classes In the same grade not using the computer for writing) . 
The assessment consists of a short (maximum 2 pages) composition followed by 
11 multiple choice items. It should be administered to the Project and 
Comparison Groups during the same week of school, just prior to the start of 
the Project. Students should be given one class period to complete the 
assessment. The teacher should Instruct students to complete the 
identifying information, read the directions, and then answer all student 
questions prior to their beginning work* 

When completed, assessments should be placed in the enclosed envelope (s) 
or box and returned for scoring to Research for Better Schools. Trained 
scorers will be reviewing and scoring the assessments in the Spring, 
following the administration of the Post-Vriting Assessment. The 
Post**Writing Assessment will be similarly administered, but will require a 
composition addressing a different topic. The results of the assessments 
will be confidential and the data will be reported in aggregate form in a 
final Project report. Individual classes will not be identified. This 
report will be made available to all participants. 

If you have any questions, please call Dr. Francine Beyer or Dr. Russell 
Dusewicz at Research for Better Schools (215-*574-9300} . 
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DELAVABE MIDDtB SCHOOL WRITING PROJECT 
POST-WRITING ASSESSMENT 



Student Name I Datet 

Grade I Agei Sext (check one) Male Female 

Race: (check one) Asian American ^Black ^Blspanlc W hite O ther 

School Names Dietricti 
Teacheri 



DIRECTIONS FOR STUDENTS 

Your assignment during this class is to write a composition. The topic for 
the composition is on the next page. Space Is provided to plan or outline ideas 
for your composition. Your plan will not be scored. < 

When you finish your plan, read it over and then write your composition on 
the lined paper attached to these directions. You may use handwriting or 
printing when you write your composition. Be sure to write clearly so your work 
can be read easily. Only the composition on the lined paper will be scored. 

Vftien you finish working on the composition, mark your answers to the 
questions on the student survey. 

Now turn th& page and begin. 
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COMPOSITION TOPIC 2 

Think about one change that you would like to make in the torn or city in ^ich 
you live and why you vould like to make that change. Write a compoiition 
diecuasing that change. Give reaaoni why the change should be made, and explain 
how the change would benefit your town or city. Remember that you are to write 
about only ONE change. 

(You may use the space below to plan or outline ideas for your composition. 
The plan or outline will not be scored.) 
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COMPOSITION 2 

(Vrlta your final copy of the composition on the lines below.) 



IF YOU NEED MORE SPACE, CONTINUE ON THE NEXT PACE. 



STUDENT SURVEY #2 



Hime I — 

School 1^ ^ Teacher: 

Directions t The questions below are about your writing experiences. Read each 
question carefully and then fill in the circle next to the best answer. Remember 
to fill in only one circle for each question. 

1. Wtien I write something for school, I think the way I write isi 

o Very Good 
o Good 
o Fair 
o Poor 
0 Very Poor 

2. When I write something for school, teachers usually grade the way I write asi 

o Very Good 
o Good 
o Fair 
o Poor 
o Very Poor 

3. How often do you use a computer at home for writing? 

o At Least Once a Day 

o At Least Once a Week 

o About Once a Month 

0 Only Once or Twice a Year 

o Never 

4. This year, how often did you use a computer at school for writing? 

o At Least Once a Day 

o At Least Once a Week 

o About Once a Month 

o Only Once or Twice a Year 

o Never 

5« How would you rate your typing or keyboarding ability? 
0 Very Good 
o Good 
o Fair 
o Poor 
o Very Poor 

6. When you have a writing assignment, how often do you plan what you* re going 
to write and how you* re going to write it? 
o Almost Always 
o Often 
o Sometimes 
o Rarely 
o Almost Never 

PLEASE CONTINUE ON NEXT fkQZ.... 
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7. When a writing assignment is returned, do teachers require you to work on 
the paper again to improve it? 

0 Almost Always 

0 Often 

0 Sometimes 

0 Rarely 

0 Almost Never 

8. Do you enjoy working on writing assignments? 

0 Almost Always 

0 Often 

0 Sometimes 

0 Rarely 

0 Almost Never 

9. How often are you required to write a paragraph or more in your school 
assignments? 

o At Least Once a Day 

o At Least Once a Week 

o About Once a Month 

o Only Once or Twice a Year 

0 Never 

10. Do you enjoy using a computer for writing? 

0 Almost Always 
o Often 
o Sometimes 
o Rarely 

o I Haven't Used a Computer for Writing 

11. Do you have a computer at home? 

o No 
o Yes 
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DELAWARE MZDDU5 SCHCOl. W>XTZN6 PROJECT 
POST-WRITING Aii;SSSMENT 

Instructions 

Enclosed are copies of the Post-Writing Assesment to be administered to 
all students who were administered the Pre-Writing Assessment last Fall. 
This includes students participating in the Computer Writing Project and 
those students not using the computer for writing who have been selected to 
serve as a Comparison Group. The assessment consists of a short (maximum 2 
pages) composition followed by 11 multiple choice items. It should be 
administered to the Project and Comparison Groups during the same week of 
school, at the end of May. Students should be given one class period to 
complete the assessment. The teacher should instruct students to complete 
the identifying information, read the directions, and then answer all 
student questions prior to their beginning work. 

When completed, assessments should be placed in the enclosed envelope (s) 
or box and returned for scoring to Research for Better Schools. Trained 
scorers will be reviewing and scoring the Pre- and Post-Assessments after 
they have all been received. The results of the assessments will be 
confidential and the data will be reported in aggregate form in a final 
Project report. Individual classes will not be identified. This report 
will be made available to all participants. 

If you have any questions, please call Dr. Fran Beyer or Dr. Russ 
Dusewicz at Research for Better Schools (215-574-9300). 
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